The Latent Maximum Entropy Principle

Learning Mixture Models with the Latent Maximum Entropy Principle

We present a new approach to estimating mixture models based on a new inference principle we have proposed: the latent maximum entropy principle (LME). LME is different both from Jaynes’ maximum entropy principle and from standard maximum likelihood estimation. We demonstrate the LME principle by deriving new algorithms for mixture model estimation, and show how robust new variants of the EM al...

Boltzmann Machine Learning with the Latent Maximum Entropy Principle

We present a new statistical learning paradigm for Boltzmann machines based on a new inference principle we have pro­ posed: the latent maximum entropy principle (LME). LME is different both from Jaynes' maximum entropy principle and from stan­ dard maximum likelihood estimation. We demonstrate the LME principle by deriving new algorithms for Boltzmann machine pa­ rameter estimation, and show h...

Latent Maximum Entropy Principle for Statistical Language Modeling

In this paper, w dcscrihe a unified probabilistic framework for statistical lmiguage modeling, latent mmmum mtmpy Wnciple. The salient feature of this approach is that the hidden causal hierarchical dependency structure can be enc d e d into thpsiatwtical model in a principled way hy mixturts of expoocntial families with a rirh expressive power. We first show tlie problem lormulatiun, whition, ...

The Latent Maximum Entropy

The Relevance of Maximum Entropy Production Principle and Maximum Information Entropy Principle in Biology

We start this talk posing the question, is there any physical principle that can serve as a selection principle in biology too? One of the first undertakings in this direction, conducted by Prigogine and Wiame [1] noticed correctly that biological processes are irreversible and as such should be described within irreversible thermodynamics. Since irreversible processes are characterized by entr...